## Problem

Custom user agent from useragententry parameter wasn't included in
connector service HTTP requests for feature flag retrieval.

 ## Root Cause

Method execution order issue in UserAgentManager.setUserAgent() - custom
user agent was set AFTER the connector service request was made.

 ## Solution

Reordered the method to set custom user agent before calling
getClientUserAgent() (which triggers feature flag fetch).

## Testing

  - Added testCustomUserAgentIncludedBeforeClientTypeEvaluation() test
  
  NO_CHANGELOG=true

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

…action preview (#1176)

## Summary
- Changes default value of `IgnoreTransactions` parameter from `0` to
`1`, making transactions disabled by default
- Updates `supportsTransactions()` to respect the `IgnoreTransactions`
flag, returning `false` when transactions are ignored (default) and
`true` when explicitly enabled
- Adds test case for when transactions are explicitly enabled via
`IgnoreTransactions=0`

## Background
The multi-statement transaction feature is currently in private preview
for limited workspaces. When BI tools (Tableau, Power BI, DBeaver)
detect transaction support via `supportsTransactions()`, they
automatically use transaction methods, causing failures for customers
not enrolled in the preview.

This change prevents unexpected failures for non-preview customers while
allowing preview participants to opt-in by explicitly setting
`IgnoreTransactions=0` in their connection string.

## Migration Path
- **Non-preview customers**: No action required - transactions are now
disabled by default
- **Preview participants**: Set `IgnoreTransactions=0` in connection
string to enable transaction support
- **GA migration**: When multi-statement transactions reach GA, flip the
default back to `0`

## Test plan
- [ ] Verify existing tests pass
- [ ] Verify default connection returns `supportsTransactions() = false`
- [ ] Verify connection with `IgnoreTransactions=0` returns
`supportsTransactions() = true`
- [ ] Verify transaction methods (`setAutoCommit`, `commit`, `rollback`)
are no-ops by default

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

## Description
Bump version to 3.1.1

### Problem
When executing queries that return 0 rows (e.g., `WHERE 1=0`), complex
types (ARRAY, MAP, STRUCT) showed only generic type names instead of
detailed type information:

  **Before:**
  - `ARRAY` instead of `ARRAY<INT>`
  - `MAP` instead of `MAP<STRING,STRING>`
  - `STRUCT` instead of `STRUCT<field: TYPE>`

  **After:**
  - Detailed type information is correctly preserved for all row counts

  ### Root Cause
In `AbstractArrowResultChunk.java`, Arrow field metadata was only
extracted inside the `while(arrowStreamReader.loadNextBatch())` loop.
For queries with 0 rows, no batches are loaded, so the loop never
executes and metadata is never extracted.

**Code location:**
`/src/main/java/com/databricks/jdbc/api/impl/arrow/AbstractArrowResultChunk.java:338-359`

  ### Solution
Extract metadata from `VectorSchemaRoot` immediately after obtaining it,
**before** the `loadNextBatch()` loop.

The Arrow IPC format always sends the schema message first (before any
record batches), so field metadata is available even when there are 0
rows. `VectorSchemaRoot` contains field vectors with metadata regardless
of row count.

  **Key changes:**
  1. Moved metadata extraction from inside the while loop to before it
2. Added defensive null checks for `VectorSchemaRoot` and field vectors
  3. Added debug logging to track metadata extraction

  ### Testing

  #### Unit Test Coverage
- ✅ Added `testMetadataExtractionWithZeroRows()` to
`ArrowResultChunkTest`
  - ✅ Verifies Arrow field metadata is extracted correctly with 0 rows
  - ✅ Tests complex types: `ARRAY<INT>`, `MAP<STRING,STRING>`
  - ✅ All 2,693 unit tests pass

  #### Manual Verification
  Tested with queries returning 0 rows:
  ```sql
  SELECT array_col, map_col, struct_col
  FROM table
  WHERE 1=0

  Result: Metadata now correctly shows detailed type information

  Impact

  - Scope: Both SQL Exec API and Thrift Server (shared code path)
- Risk: Low - backward compatible change, only affects metadata
extraction timing
  - Benefits:
    - Fixes schema discovery for WHERE 1=0 pattern
    - Improves metadata availability for empty result sets
    - Aligns with Arrow IPC specification behavior

  Additional Context

- Arrow IPC specification guarantees schema is sent before record
batches
- VectorSchemaRoot.getFieldVectors() is available immediately after
ArrowStreamReader.getVectorSchemaRoot()
- No performance impact: metadata extraction is now done once upfront
instead of conditionally on first batch

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

## Description
<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->
This PR introduces lazy loading support for inline Arrow results to
improve memory efficiency when handling large result sets.

Previously, InlineChunkProvider would eagerly fetch all arrow batches
upfront when results had hasMoreRows = true, which could lead to memory
issues with large datasets. This change splits the handling into two
separate paths:
1. Lazy path (new): For Thrift-based inline Arrow results (when
ARROW_BASED_SET is returned), we now use LazyThriftInlineArrowResult
which fetches arrow batches on-demand as the client iterates through
rows. This is similar to how LazyThriftResult works for columnar data.
2. Remote path (existing): For URL-based Arrow results (URL_BASED_SET),
we continue using ArrowStreamResult with RemoteChunkProvider which
downloads chunks from cloud storage.

The InlineChunkProvider is now only used for SEA results with JSON_ARRAY
format and INLINE disposition (contain all data inline {no hasMoreRows
flag set}).

This will reduce memory consumption and improve performance when dealing
with large inline Arrow result sets similar to
#975.

## Testing
<!-- Describe how the changes have been tested-->
- Unit tests
- Integration tests
- Manual testing

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->
Bypassing an existing failure on CI/CD because of 3e4f21c

…istency (#1182)

## Summary
This PR adds TIMESTAMP_NTZ normalization in the Thrift path to ensure
consistent metadata behavior across both SEA and Thrift API paths.

## Background
PR #1177 moved Arrow metadata extraction earlier in the processing
pipeline, which exposed an inconsistency: the Thrift path started
returning the correct "TIMESTAMP_NTZ" from server metadata, while the
SEA path was already normalizing it to "TIMESTAMP" for backward
compatibility.

## Changes
- Added TIMESTAMP_NTZ → TIMESTAMP normalization in
`DatabricksResultSetMetaData.java` Thrift constructor (lines 205-208)
- This brings Thrift path behavior in line with existing SEA path
normalization
- Fixes test failure in
`PreparedStatementIntegrationTests.testGetMetaData_NoResultSet`

## Testing
- ✅ Local test run:
`PreparedStatementIntegrationTests.testGetMetaData_NoResultSet` passes
- ✅ Metadata now consistent before and after `executeQuery()` for
TIMESTAMP_NTZ columns
- ✅ Both SEA and Thrift paths return "TIMESTAMP" for TIMESTAMP_NTZ
columns

## Related
- Builds on PR #1177 (Fix Arrow field metadata not available for queries
with 0 rows)
- Fixes issue introduced by early metadata extraction in PR #1177
- Maintains backward compatibility with existing behavior

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

…oad parameter (#1183)

## Summary

Add support for disabling CloudFetch via `EnableQueryResultDownload=0`
connection parameter to use inline Arrow results instead.

## Changes

- Add `isCloudFetchEnabled()` method to `IDatabricksConnectionContext`
interface
- Implement the method in `DatabricksConnectionContext` using existing
`EnableQueryResultDownload` parameter
- Update `DatabricksThriftServiceClient` to respect this setting when
making execute requests
- Add unit tests for the new functionality

## Usage

To disable CloudFetch and use inline Arrow results:

```
jdbc:databricks://host:port/default;EnableQueryResultDownload=0;...
```

---------

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

…lemetry code audit comments- part1 (#1163)

## Description
- 4 things that is improved with respect to telemetry : 
- Common object mapper across telemetry use-case (This is already thread
safe and is expensive to create, i.e., good tor re-use)
- Make`flushIntervalMillis` config same across both telemetry clients
(un-auth and auth)
- Clear connection param cache when connection is closed : this was a
memory leak before
- Rather than creating a scheduledExecutor for each telemetry client, we
share it across a factory.

## Testing
- unit tests

## Additional Notes to the Reviewer
- When the LAST connection to a host is closed, all pending telemetry
events for that host are flushed across all prior connections (since
they all shared the same TelemetryClient). i.e., If you have 5
connections to `host-A`, closing connections 1-4 does nothing (just
decrements refCount). Only when you close connection 5 (the last one)
does the flush occur, sending all accumulated telemetry from all 5
connections.

NO_CHANGELOG=true

## Description

This PR enhances geospatial datatype handling to include SRID (Spatial
Reference System Identifier) information in column type names and fixes
multiple issues related to complex datatype handling across different
result formats.

### Key Changes

1. **Geospatial Type Name Enhancement**
- Column type names now include SRID: `GEOMETRY(4326)` instead of
`GEOMETRY`
   - Applies to both GEOMETRY and GEOGRAPHY types
- Preserves full type information in metadata for better type
identification

2. **SEA Inline Mode Complex Type Fix**
- Fixed issue where complex types (ARRAY, MAP, STRUCT) were not returned
as complex objects in SEA Inline mode (JSON array result format)
- Now properly converts to complex datatype objects when
`EnableComplexDatatypeSupport=true`

3. **Thrift CloudFetch Metadata Enhancement**
- Fixed error when extracting type details (e.g., `INT` from
`ARRAY<INT>`) in Thrift CloudFetch mode
- Enhanced `getColumnInfoFromTColumnDesc()` to use Arrow schema metadata
alongside `TColumnDesc`
- Arrow schema provides complete type information (e.g., `ARRAY<INT>`)
while `TColumnDesc` only contains base type (e.g., `ARRAY`)

4. **Arrow Metadata Extraction**
- Added `DatabricksThriftUtil.getArrowMetadata()` to deserialize Arrow
schema from `TGetResultSetMetadataResp`
- Fixed null arrow metadata issue in `DatabricksResultSet` constructor
for Thrift CloudFetch mode

## Testing

### Unit Tests
- All existing unit tests pass and additional tests are added for new
methods

### Integration Tests
- `GeospatialTests.java` - Comprehensive E2E integration test
  - Tests geospatial types (GEOMETRY and GEOGRAPHY)
  - Validates **24 configuration combinations**:
    - Protocol: Thrift / SEA
    - Serialization: Arrow / Inline
- CloudFetch: Enabled / Disabled (only with Arrow, as CloudFetch
requires Arrow)
    - GeoSpatial Support: Enabled / Disabled
    - Complex Type Support: Enabled / Disabled
  - Validates metadata: column types, type names, class names
  - Validates values: WKT representation, SRID
- Validates behavior when geospatial objects are enabled vs. disabled
(STRING fallback)
  - **All 24 tests pass** ✅

## Additional Notes to the Reviewer

Other required details are mentioned in comments in the diff

---------

Signed-off-by: Sreekanth Vadigi <sreekanth.vadigi@databricks.com>

## Summary

Implements proactive prefetching with a sliding window for both Thrift
columnar and inline Arrow results, eliminating blocking at batch
boundaries and improving throughput.

## Key Components

### New Streaming Infrastructure
- **`ThriftStreamingProvider<T>`**: Generic type-safe streaming provider
with background prefetch thread and configurable sliding window
- **`StreamingBatch<T>`**: Type-safe batch container with lifecycle
management and error handling
- **`ThriftResponseProcessor<T>`**: Interface for pluggable response
processors
  - `ColumnarResponseProcessor`: Processes Thrift columnar results
- `InlineArrowResponseProcessor`: Processes inline Arrow results with
schema caching

### Result Implementations
- **`StreamingInlineArrowResult`**: High-throughput streaming
implementation for inline Arrow results with background prefetching
- **`StreamingColumnarResult`**: Streaming implementation for Thrift
columnar results with prefetch

### Supporting Classes
- **`ThriftBatchFetcher`** / **`ThriftBatchFetcherImpl`**: Abstraction
for fetching batches from the Thrift server

<img width="1792" height="1234" alt="streaming inline"
src="https://github.com/user-attachments/assets/66ea9b83-a16b-42d5-9280-cb1fb81dadeb"
/>

## Configuration

| Parameter | Description | Default |
|-----------|-------------|---------|
| `EnableInlineStreaming` | Toggle streaming mode for inline results |
`1` (enabled) |
| `ThriftMaxBatchesInMemory` | Sliding window size (max batches kept in
memory) | `3` |

## Key Features

1. **Background Prefetching**: Dedicated thread fetches batches ahead of
consumption
2. **Sliding Window**: Configurable memory limit prevents unbounded
memory growth
3. **Type Safety**: Generic `ThriftStreamingProvider<T>` eliminates
unsafe casting
4. **Graceful Error Handling**: 
   - Try-catch around resource cleanup to prevent cascading failures
   - Timeout on batch creation wait to prevent indefinite blocking
5. **Comprehensive Logging**: Debug/error logging for troubleshooting

## Testing

- Updated `ExecutionResultFactoryTest` for new factory logic
- Updated `DatabricksThriftServiceClientTest` for CloudFetch control
- Existing integration tests cover streaming behavior

## Usage

Streaming is enabled by default. To disable and use lazy loading
instead:

```
jdbc:databricks://host:port/default;EnableInlineStreaming=0;...
```

To adjust the sliding window size:

```
jdbc:databricks://host:port/default;ThriftMaxBatchesInMemory=5;...
```

---------

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

…ents (#1186)

## Description

Fixed `IndexOutOfBoundsException` that occurs when executing DDL
statements (e.g., `CREATE DATABASE`) using the Thrift protocol. The bug
manifests when there's a mismatch between the number of Thrift column
descriptors and Arrow schema fields.

### Root Cause

When executing DDL statements, the Databricks server behavior is:
- **Thrift Protocol**: Returns column descriptors including a "Result"
status column (1 column)
- **Arrow Schema**: Returns an empty schema with 0 fields (no actual
data)
- **The Bug**: Code attempted to access `arrowMetadata[0]` without
checking if the list was empty

This mismatch caused `IndexOutOfBoundsException` when the driver tried
to access arrow metadata at index 0 of an empty list.

### Debug Evidence

**TColumnDesc (Thrift)**:
```
Column[0]:
  name: Result
  type: STRING_TYPE
  position: 1
  Full TColumnDesc: TColumnDesc(columnName:Result, typeDesc:TTypeDesc(...), position:1, comment:)
```

**Arrow Schema**:
```
Arrow schema bytes length: 72
Deserialized Arrow schema, field count: 0  ← Empty!
Arrow metadata list: size=0
```

### Changes Made

Added bounds checking in two locations where arrow metadata is accessed:

1. **`ArrowUtil.java:247`** - Used by `StreamingInlineArrowResult`
2. **`DatabricksResultSetMetaData.java:195`** - Used for result set
metadata construction

**Before:**
```java
String columnArrowMetadata =
    arrowMetadata != null ? arrowMetadata.get(columnIndex) : null;
```

**After:**
```java
String columnArrowMetadata =
    arrowMetadata != null && columnIndex < arrowMetadata.size()
        ? arrowMetadata.get(columnIndex)
        : null;
```

## Testing

### Manual Testing

**Test Case**: Execute CREATE DATABASE statement
```java
String sqlQuery = "CREATE DATABASE IF NOT EXISTS hive_metastore.test_db";
boolean hasResultSet = stmt.execute(sqlQuery);
```


**Before Fix**: `IndexOutOfBoundsException: Index 0 out of bounds for
length 0`
**After Fix**: Executes successfully, returns `hasResultSet=false`

## Additional Notes to the Reviewer

NO_CHANGELOG=true

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

#1181)

## Description
<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->
Modified the logic for enableMultipleCatalogSupport parameter to only
return results when the catalog provided in metadata calls is null or is
equal to the current catalog when the param is disabled. This matches
the behaviour with existing driver.
## Testing
<!-- Describe how the changes have been tested-->
Tested locally 
## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

## Description
Implements NonRowcountQueryPrefixes flag to match exiting JDBC driver
behavior. This allows users to specify comma-separated query prefixes
(like INSERT, UPDATE, DELETE) that should return result sets instead of
row counts.

Changes:
- Added NON_ROWCOUNT_QUERY_PREFIXES parameter to DatabricksJdbcUrlParams
- Added getNonRowcountQueryPrefixes() method to connection context
interface and implementation
- Updated shouldReturnResultSet() logic to check configured prefixes
before SQL patterns
- Added 11 comprehensive unit tests covering various scenarios
- Updated NEXT_CHANGELOG.md with feature description

Usage:
NonRowcountQueryPrefixes=INSERT,UPDATE,DELETE,MERGE

## Testing
Tests: All 75 tests pass (64 existing + 11 new)


<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->


<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

… new resultset (#1187)

Added support for getClientInfoProperties and getTypeInfo to return a
new resultset and not return the same resultset matching the JDBC spec

## Description
<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->
Added support for getClientInfoProperties and getTypeInfo to return a
new resultset and not return the same resultset matching the JDBC spec
## Testing
<!-- Describe how the changes have been tested-->
Added tests
## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

Fixes: #1178

## Description
- Earlier the error telemetry logs only contained 
  - `error_name: "AUTH_ERROR"
stack_trace: "Failed to retrieve the exchanged token"`. 
- With this change, it would be more descriptive - i.e., contain http
error code and reason.
- `Failed to retrieve the exchanged token from OIDC token endpoint. HTTP
request failed by code: 401, status line: HTTP/1.1 401 Unauthorized.
Error message: Invalid credentials.`

## Testing
<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

## Description
<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->

## Testing
<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer

NO_CHANGELOG=true

## Description
- The request ID is typically in the response header; this PR adds it to
the error message for easier debugging.

## Testing
- Manually testing BYOT flow error scenario 

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

## Description
- This fix was made in the [PR
#1135](#1135),
but it did not consider the fetch result timeout. This PR fixes that.

## Testing
- unit tests

## Additional Notes to the Reviewer
- Most of the code changes are propogating errors as `SQLException`, but
the main code change is a small one and is present in
`src/main/java/com/databricks/jdbc/common/util/DatabricksThriftUtil.java`
file

OVERRIDE_FREEZE=true

…L execution (#1205)

## Description
Following changes in JDBC driver to honour the JDBC spec:

1. Post-execution validation in executeQuery() and executeUpdate()
   - Validation happens AFTER execution to match existing driver
   - executeQuery() validates result should be a ResultSet
   - executeUpdate() validates result should be an update count
   - Both throw SQLException if mismatch detected

2. JDBC-compliant getResultSet() behavior
   - Returns null for DML statements (when execute() returns false)
   - Returns ResultSet only when execute() returns true
   - Caches shouldReturnResultSet to avoid recomputation

3. Lazy update count evaluation
   - Avoids iterating result rows until getUpdateCount() is called
   - Uses UPDATE_COUNT_NOT_YET_EVALUATED sentinel value (-2L)
   - Improves performance for statements that don't need update count

4. PreparedStatement optimization
   - Caches shouldReturnResultSet at construction time
   - SQL is known upfront, no need to recompute on each execution

NonRowcountQueryPrefixes makes a DML behave as a query. E.g. behavior
with NonRowcountQueryPrefixes=INSERT:
- execute(INSERT) returns true, provides ResultSet, getUpdateCount() =
-1
- executeQuery(INSERT) succeeds and returns ResultSet
- executeUpdate(INSERT) throws SQLException after execution
- Other DML (UPDATE/DELETE) and SELECT remain unaffected

## Testing
Test results:
- All 76 unit tests in DatabricksStatementTest pass
- DML test: getResultSet() returns null for DML statements
- NonRowcountQueryPrefixes test: all behaviors match existing driver
- Execution order test: validates after execution (not before)


<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->


<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

OVERRIDE_FREEZE=true

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>

…esultSet (#1223)

## Summary

- Cache the `TelemetryCollector` reference at `DatabricksResultSet`
construction time instead of resolving it on every `next()` call
- Eliminates per-row overhead introduced by PR #1163 (telemetry refactor
from singleton to per-connection)
- Restores Arrow cloud-fetch throughput to baseline levels

## Problem

PR #1163 refactored telemetry collection from a singleton pattern to
per-connection, but inadvertently introduced per-row overhead in
`DatabricksResultSet.next()`. On every row iteration (~6.5M times for a
1025MB query), the code now traverses:

```
parentStatement.getStatement().getConnection()
  → cast to DatabricksConnection
  → getConnectionContext()
  → TelemetryCollectorManager.getInstance().getOrCreateCollector(context)
  → ConcurrentHashMap.computeIfAbsent()
```

This adds ~2.3 microseconds per row, compounding to ~15 seconds of
overhead over 6.5M rows.

## Fix

Resolve the `TelemetryCollector` once in each constructor via a
`resolveTelemetryCollector()` helper and store it in a `final` field.
The `next()` method calls the cached collector directly, bypassing the
entire per-row resolution chain. The connection-to-collector mapping is
stable for the lifetime of a result set, so caching at construction time
is safe.

**Before:**
```java
public boolean next() throws SQLException {
  checkIfClosed();
  boolean hasNext = this.executionResult.next();
  TelemetryHelper.recordResultSetIteration(
      parentStatement, statementId, resultSetMetaData.getChunkCount(), hasNext);
  return hasNext;
}
```

**After:**
```java
public boolean next() throws SQLException {
  checkIfClosed();
  boolean hasNext = this.executionResult.next();
  if (cachedTelemetryCollector != null) {
    cachedTelemetryCollector.recordResultSetIteration(
        statementId.toSQLExecStatementId(), resultSetMetaData.getChunkCount(), hasNext);
  }
  return hasNext;
}
```

## Benchmark Results

### Regression Bisection (1025MB query, ~6.5M rows, Arrow cloud-fetch,
Thrift mode, warm iteration)

| Build | Commit | Fetch (ms) | Rows/s | Per-row cost | Status |
|-------|--------|-----------|--------|-------------|--------|
| v3.1.1 (baseline) | `0e8b1ad` | 5,037 | 1.29M | 0.78 us | ✅ Baseline |
| Pre-#1163 | `9c9ffe2` | 4,720 | 1.38M | 0.73 us | ✅ Matches baseline |
| Post-#1163 | `8a2f333` | 19,727 | 329K | 3.04 us | ❌ 3.9x regression |
| v3.1.2 HEAD (unfixed) | `47235a1` | 20,382 | 318K | 3.14 us | ❌ 4.0x
regression |
| **v3.1.2 (this fix)** | `eaa3f64` | **4,447** | **1.46M** | **0.68
us** | ✅ **12% faster than baseline** |

### Post-Fix Segment Latency (500K rows per segment, warm iteration)

The fix restores the expected **bursty** pattern (fast in-memory
iteration alternating with chunk-download waits):

```
 500K rows:   73ms    (in-memory, fast)
   1M rows:  393ms    (chunk wait)
 1.5M rows:  147ms    (in-memory)
   2M rows:  317ms    (chunk wait)
 2.5M rows:  409ms    (chunk wait)
   3M rows:  282ms    (mixed)
 3.5M rows:  250ms    (mixed)
   4M rows:  314ms    (chunk wait)
 4.5M rows:  197ms    (in-memory)
   5M rows:  370ms    (chunk wait)
 5.5M rows:  361ms    (chunk wait)
   6M rows:  146ms    (in-memory)
```

vs the unfixed v3.1.2 which showed flat ~1,450ms per segment (dominated
by uniform per-row overhead).

### Comprehensive Comparison (1025MB, warm iteration)

| Driver | Mode | Arrow | Fetch (ms) | Rows/s |
|--------|------|-------|-----------|--------|
| Simba 2.7.6 | Thrift | N/A | 109,994 | 59.0K |
| OSS v3.1.1 | Thrift | Yes | 5,037 | 1.29M |
| OSS v3.1.1 | SEA | Yes | 4,202 | 1.54M |
| OSS v3.1.1 | Thrift | No | 163,051 | 39.8K |
| OSS v3.1.2 (unfixed) | Thrift | Yes | 20,382 | 318K |
| OSS v3.1.2 (unfixed) | SEA | Yes | 19,882 | 326K |
| OSS v3.1.2 (unfixed) | Thrift | No | 145,131 | 44.7K |
| **OSS v3.1.2 (this fix)** | **Thrift** | **Yes** | **4,447** |
**1.46M** |

NoArrow mode is unaffected by the regression (~145-163s for both
versions) because the per-row telemetry overhead (~2.3us) is negligible
compared to the ~25us/row cost of non-Arrow iteration.

## Test plan

- [x] `mvn test -Dtest=DatabricksResultSetTest` — all 48 tests pass (43
existing + 5 new)
- [x] New tests cover: next() with cached telemetry, null
parentStatement (no NPE), exception handling in resolution, null
connection context, collector resolved once at construction
- [x] Benchmark verified: fetch time drops from 20,382ms → 4,447ms (4.6x
improvement)
- [ ] CI pipeline

🤖 Generated with [Claude Code](https://claude.com/claude-code)

NO_CHANGELOG=true
OVERRIDE_FREEZE=true

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

## Summary
Adds new unit tests and extends existing ones for auth-related classes.
Aligns new tests with repo conventions (parameterized tests,
TestConstants, DRY).

## Changes

### New test classes
- **AuthConstantsTest** – Covers `AuthConstants` (grant_type keys) via
parameterized test and non-empty checks.
- **AzureMSICredentialsTest** – Covers `AzureMSICredentials`:
system/user-assigned identity, management endpoint token, caching,
clientId handling (parameterized), error paths (HTTP/parsing/IO),
refresh token and expiry.

### Extended test classes
- **DatabricksTokenFederationProviderTest** – getCredentialsProvider,
different-host token exchange, authType.
- **EncryptedFileTokenCacheTest** – Nested path creation, overwrite,
no-refresh-token, expired token, long token, corrupted/empty file,
save/load cycles, token types (parameterized).
- **JwtPrivateKeyClientCredentialsTest** – Scopes, null
clientId/keyFile/kid, invalid key file, EC key (ES256), token caching.
- **OAuthRefreshCredentialsProviderTest** – getToken before configure,
token caching, authType, Authorization header format.
- **PrivateKeyClientCredentialProviderTest** – authType, build with all
params, configure/HeaderFactory, JWT algorithm, null scope.

### Test quality
- Use of **TestConstants** where applicable (e.g.
`AzureMSICredentialsTest`).
- **Parameterized tests** in AuthConstantsTest, AzureMSICredentialsTest
(clientId), EncryptedFileTokenCacheTest (token types).
- **DRY**: mock helper in AzureMSICredentialsTest; removed duplicate
test in PrivateKeyClientCredentialProviderTest; removed test that only
asserted on SDK `Token` in DatabricksTokenFederationProviderTest.

## Testing
- `mvn test
-Dtest="AuthConstantsTest,AzureMSICredentialsTest,DatabricksTokenFederationProviderTest,EncryptedFileTokenCacheTest,JwtPrivateKeyClientCredentialsTest,OAuthRefreshCredentialsProviderTest,PrivateKeyClientCredentialProviderTest"`
passes.
NO_CHANGELOG=true

---------

Signed-off-by: Nikhil Suri <nikhil.suri@databricks.com>

## Description
There are 2 fixes in this : 
1. We are piggy-backing onto the existing HTTP timeouts and retries for
telemetry. We can configure socket timeout to be separate for telemetry
- something smaller.
2. There is a large number of INVALID_COLUMN_INDEX logs- this is a
fallback in metadata calls today. I will be configuring this to have a
lower telemetry export level

## Testing
<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer
-  need not be in changelog. 
NO_CHANGELOG=true 
OVERRIDE_FREEZE=true

## Description
- new version cut : 3.2.1
- Also allow manual trigger of jdk8 action from the github ui

## Testing
- [Internal testing doc
link](https://docs.google.com/document/d/14-PQukvFixRS6P4cfCPT8oCjKAY1El3JoMuD7fPLx3g/edit?tab=t.0)

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

OVERRIDE_FREEZE=true

## Summary
- Adds a `CLAUDE.md` file documenting build/test commands, project
structure, and coding conventions
- Captures common PR check pitfalls: DCO sign-off, NEXT_CHANGELOG /
`NO_CHANGELOG=true`, release freeze override, and GitHub account
selection (non-EMU)
- Removes release freeze after 3.2.1 release
(`development/.release-freeze.json` set to `freeze: false`)

NO_CHANGELOG=true

## Test plan
- [ ] Verify CLAUDE.md renders correctly on GitHub
- [ ] Confirm CI checks pass (DCO, changelog)
- [ ] Verify release freeze is disabled after merge

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…ses (#1206)

## Summary
- **rollback() now throws** `DatabricksTransactionException` when called
while the connection is in auto-commit mode (no active transaction),
aligning behavior with the JDBC spec
- **fetchAutoCommitStateFromServer()** now accepts both `"1"`/`"0"` and
`"true"`/`"false"` responses from the `SET AUTOCOMMIT` query, since
different server implementations return different formats
- Updated E2E and unit tests to reflect the corrected behavior

## Context
1. `connection.rollback()` in auto-commit mode should throw an
exception, not silently succeed
2. `SET AUTOCOMMIT` query returns `"1"`/`"0"` rather than
`"true"`/`"false"` depending on the server

## Test plan
- [x] Unit tests pass: `DatabricksConnectionTest` (42/42 pass)
- [ ] E2E tests: `TransactionTests` and
`ExplicitTransactionStatementTests` against staging warehouse

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

## Summary
- Removes two unused `TelemetryHelper.recordResultSetIteration`
overloads that became dead code after #1223 switched
`DatabricksResultSet.next()` to use `cachedTelemetryCollector` directly
- Removes the now-unused `DatabricksConnection` import
- Supersedes #1210

NO_CHANGELOG=true

## Test plan
- [x] `mvn compile` passes
- [ ] CI checks pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

## Summary
- Adds structured `debug`-level logs across the cloud fetch pipeline
(chunk download, link wait, decompression, task completion) with
`statementId` correlation for end-to-end tracing
- Enriches the `executeStatement` completion log with manifest details
(format, totalBytes, totalChunks, totalRows, compression) replacing the
previous sparse log
- Breaks down `ArrowResultChunk.downloadData` timing into HTTP response,
stream read, and decompress+parse phases

NO_CHANGELOG=true

## Test plan
- [ ] Verify logs appear when debug logging is enabled (e.g.
`log4j.logger.com.databricks.jdbc=DEBUG`)
- [ ] Verify no logs appear at default (INFO) level — no noise for
customers
- [ ] Run existing unit tests to confirm no regressions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…#1227)

getObject(int, Class<T>) was missing a null guard before passing objects
to converters, causing NPE in DateConverter.toDate() and all other
converters when used by Spring DataClassRowMapper, Hibernate, etc. Added
null check in both DatabricksResultSet and ConverterHelper as
defense-in-depth, consistent with getObject(int) and getObject(int,
Map).

## Description
<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->

## Testing
<!-- Describe how the changes have been tested-->
Added unit tests

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

NO_CHANGELOG=true

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

## Summary
- Metadata SHOW commands routed through `executeStatement()` via
`UseQueryForMetadata` were incorrectly executing asynchronously because
`getRequest()` always forces `runAsync=true` in production
- Added `StatementType` parameter to `getRequest()` and skip the
`runAsync=true` override when `statementType == METADATA`, aligning
Thrift client behavior with the SEA client
- Updated existing test expectations and added new tests verifying
`runAsync` is `false` for METADATA and `true` for SQL statements

## Test plan
- [x] `DatabricksThriftServiceClientTest` — 43 tests pass (includes 2
new tests)
- [x] `DatabricksMetadataQueryClientTest` — 44 tests pass
- [x] `DatabricksSessionTest` — 17 tests pass
- [ ] Verify metadata SHOW commands execute synchronously in production
with `UseQueryForMetadata` enabled

🤖 Generated with [Claude Code](https://claude.com/claude-code)

NO_CHANGELOG=true

---------

Signed-off-by: Gopal Lal <gopal.lal@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

## Summary
- Fixes #1247: Date fields within `ARRAY<STRUCT>`, `ARRAY<DATE>`,
`MAP<*,DATE>`, and other complex types were serialized as epoch day
integers instead of proper `java.sql.Date` objects.
- Arrow's `getObject()` on nested types returns epoch day integers for
DATE fields. `ComplexDataTypeParser.convertPrimitive()` now falls back
to parsing epoch day integers via `LocalDate.ofEpochDay()` when
`Date.valueOf()` fails on non-ISO-8601 input.
- Non-numeric invalid date strings preserve the original
`IllegalArgumentException`.

## Test plan
- [x] `testDateAsEpochDayInStruct` — verifies DATE epoch day integers in
`ARRAY<STRUCT<event_date:DATE>>`
- [x] `testDateAsEpochDayInArray` — verifies DATE epoch day integers in
`ARRAY<DATE>`
- [x] `testDateAsEpochDayInMap` — verifies DATE epoch day integers in
`MAP<STRING,DATE>`
- [x] `testDateAsStringInStruct` — verifies ISO-8601 date strings still
work (no regression)
- [x] `testInvalidDateStringInStructThrowsOriginalException` — verifies
error behavior for invalid strings
- [x] All 18 `ComplexDataTypeParserTest` tests pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

)

## Description

Closes #1201
Closes #1254

Goes through each character in the sql string while keeping track of the
current state
(normal, comment, string literal, identifier). Provides an interface for
consumers to iterate over the non-comment characters.

## Testing

See unit tests. I was also testing with this script:

<details>

<summary>Main.java</summary>

```java
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.Statement;
import java.util.List;

public class Main {

    public static void main(String[] args) throws Exception {
        String url = System.getenv("DATABRICKS_URL");
        if (url == null) {
            System.err.println("DATABRICKS_URL environment variable is not set");
            System.exit(1);
        }

        Class.forName("com.databricks.client.jdbc.Driver");

        List<String> examples;
        examples = List.of(
                // "/* This is a comment */",
                "SELECT 1; /* This is also a comment */",
                """
                                SELECT /* This is a comment
                                that spans multiple lines */ 1;
                                """,
                "SELECT /* Comments are not limited to Latin characters: 评论 😊 */ 1;",
                "SELECT /* Comments /* can be */ nested */ 1;",
                "SELECT /* Quotes in '/*' comments \"/*\" are not special */ */ */ 1;",
                "/* A prefixed comment */ SELECT 1;",
                """
                                SELECT '/* This is not a comment */';
                                /* This is not a comment */
                                """
        );
        for (int i = 0; i < examples.size(); i++) {
            String example = examples.get(i);
            System.out.println("==============================");
            try {
                Connection conn = DriverManager.getConnection(url);
                Statement stmt = conn.createStatement();
                ResultSet rs = stmt.executeQuery(example);
                while (rs.next()) {
                    System.out.println("Result " + i + ": " + rs.getObject(1));
                }
            } catch (Exception e) {
                System.out.println(e.getMessage());
            }
            System.out.println("==============================");
            System.out.println("");
            System.out.println("");
        }

        System.out.println("");
        System.out.println("");

        System.out.println("==============================");
        try {
            Connection conn = DriverManager.getConnection(url);
            Statement stmt = conn.createStatement();
            ResultSet rs = stmt.executeQuery("""
            /* 
            hi */
            SELECT  1;
            """);
            while (rs.next()) {
                System.out.println("Multi line block: " + rs.getObject(1));
            }
        } catch (Exception e) {
            System.out.println("Multi line block: " + e.getMessage());
        }
        System.out.println("==============================");

        System.out.println("");
        System.out.println("");

        System.out.println("==============================");
        try {
            Connection conn = DriverManager.getConnection(url);
            Statement stmt = conn.createStatement();
            ResultSet rs = stmt.executeQuery("""
            /* /* */ */
            SELECT 2;
            """);
            while (rs.next()) {
                System.out.println("Nested: " + rs.getObject(1));
            }
        } catch (Exception e) {
            System.out.println("Nested: " + e.getMessage());
        }
        System.out.println("==============================");

        System.out.println("");
        System.out.println("");

        System.out.println("==============================");
        try {
            Connection conn = DriverManager.getConnection(url);
            PreparedStatement pstmt = conn.prepareStatement("""
            /* ? /* ? */ ? */
            SELECT ? /* ? /* ? */ ? */
            /* ? /* ? */ ? */;
            """);
            pstmt.setString(1, "hello");
            ResultSet rs = pstmt.executeQuery();
            while (rs.next()) {
                System.out.println("Nested param: " + rs.getObject(1));
            }
        } catch (Exception e) {
            System.out.println("Nested param: " + e.getMessage());
        }
        System.out.println("==============================");

    }
}
```

</details>

## Additional Notes to the Reviewer

---------

Signed-off-by: rileythomp <rileythompson99@gmail.com>

…1261)

## Summary
- Thrift metadata RPCs (GetSchemas, GetTables, GetColumns, etc.) treat
catalog names as patterns, so `_` is interpreted as a single-character
wildcard. This causes `my_catalog` to incorrectly match `mycatalog`,
`my1catalog`, etc.
- Adds a `TreatMetadataCatalogNameAsPattern` connection property
(default `false`). When disabled, unescaped `_` in catalog names are
escaped with `\` before passing to Thrift requests.
- Applied to 4 metadata methods: `listSchemas`, `listTables`,
`listColumns`, `listFunctions` (Thrift RPC path)

## Test plan
- [x] Parameterized unit tests for `WildcardUtil.escapeCatalogName()`
(null, no wildcards, single/multiple underscores, already-escaped,
percent left unchanged)
- [x] `DatabricksThriftServiceClientTest`: verify `listSchemas` escapes
catalog by default, does not escape when property is `true`, and
`listCrossReferences` escapes both parent and foreign catalogs
- [ ] Manual verification with a Databricks workspace using a catalog
containing `_` in its name

NO_CHANGELOG=true

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Gopal Lal <gopal.lal@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…x coverage (#1255)

## Description
1. The JDK8 GH action is broken, this PR fixes it. 
2. This PR also adds a maven profile that does compile, run unit tests,
and package and nothing else (similar to our old `mvn clean test`)
    1. What -Plocal skips:
      - NVD/dependency-check scan
      - Arrow patch tests (org.apache.arrow.memory.*)
      - Integration & fake service tests
      - Proxy, SSL, and logging tests

## Testing
- Run `mvn -pl jdbc-core -Plocal clean test`
- For packaging, `mvn -pl jdbc-core,assembly-uber install -Plocal`

## Additional Notes to the Reviewer
Note : arrow specific tests are not relevant in jdk8 branch as it is
specific for JDK16+


NO_CHANGELOG=true

---------

Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>

…or columns with no default (#1270)

## Summary

- `COLUMN_DEF` (column 13 of `getColumns()`) was returning the column's
**type name** instead of `null` for columns with no default value,
violating the JDBC spec.
- Root cause: `COLUMN_DEF_COLUMN` was mapped to the `"columnType"`
result-set field, so both the SQL path (default switch case) and the
Thrift path (explicit `case "COLUMN_DEF"`) returned the type name.
- Neither `SHOW COLUMNS` nor the Thrift `TGetColumns` RPC exposes column
default values, so `COLUMN_DEF` must always be `null` for now.
- Fix: added `COLUMN_DEF_COLUMN` to `NULL_COLUMN_COLUMNS` (handles the
Thrift path) and added an explicit `case "COLUMN_DEF": object = null` in
`getRows()` (handles the SQL path).

## Test plan

- added uts
- Tested with legacy driver too : we return null only if it is null, we
don't have a way to determine if default has been set

```
  Driver: DatabricksJDBC v02.07.06.1023
  id:          TYPE_NAME=INT,     COLUMN_DEF=null
  name:        TYPE_NAME=STRING,  COLUMN_DEF=null
  description: TYPE_NAME=VARCHAR, COLUMN_DEF=null
  status:      TYPE_NAME=STRING,  COLUMN_DEF=null  ← has DEFAULT 'active'
  score:       TYPE_NAME=INT,     COLUMN_DEF=null  ← has DEFAULT 0
```

Test : 

```
          String catalog = "samikshya-catalog";
          String schema = "newschema";
          String table = "simba_test_column_def";

 con.createStatement().execute(
                  "CREATE TABLE IF NOT EXISTS `" + catalog + "`.`" + schema +
  "`.`" + table + "`"
                  + " (id INT NOT NULL, name STRING NOT NULL, description
  VARCHAR(255),"
                  + "  status STRING NOT NULL DEFAULT 'active', score INT
  DEFAULT 0)"
                  + " TBLPROPERTIES('delta.feature.allowColumnDefaults' =
  'supported')");

          try (ResultSet rs = con.getMetaData().getColumns(catalog, schema,
  table, null)) {
              while (rs.next()) {
                  System.out.println(rs.getString("COLUMN_NAME")
                          + ": TYPE_NAME=" + rs.getString("TYPE_NAME")
                          + ", COLUMN_DEF=" + rs.getString("COLUMN_DEF"));
              }
          }

          con.createStatement().execute("DROP TABLE IF EXISTS `" + catalog +
  "`.`" + schema + "`.`" + table + "`");
          con.close();
      }
  }
```

Closes #1267

---------

Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>

…#1274)

## Summary

- Adds `.claude/commands/sync-jdk8-branch.md`, a new slash command
(`/sync-jdk8-branch`) for syncing the `jdk-8` branch with the latest
`main`
- Also update the jdk8 actions, as the branch is to be updated to the
latest multimodule setup
- PRs created by this command target **upstream
(`databricks/databricks-jdbc`) `jdk-8` branch**, not a fork

## Test plan

Battle tested `/sync-jdk8-branch` which produced PR
[#1275](#1275). Added
more changes to the slash command while using it



NO_CHANGELOG=true

---------

Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

…ndle (#1273)

## Summary
- **Fix infinite poll loop on invalid handle**:
`checkOperationStatusForErrors()` now checks `TStatus.statusCode` for
`INVALID_HANDLE_STATUS` during polling. Previously only `operationState`
was checked — if the server returned `INVALID_HANDLE_STATUS` without
setting `operationState` (e.g. after a driver restart), the polling loop
ran indefinitely.
- **Add timeout and sleep to metadata polling loop**:
`fetchMetadataResults()` previously had no timeout and no sleep between
polls. It now uses a configurable `MetadataOperationTimeout` (default
300s) and sleeps between polls using the same interval as the SQL
execution polling loop.
- **New connection property**: `MetadataOperationTimeout` (seconds,
default 300, 0 = no timeout) controls the metadata polling timeout.

## Context
ES-1774740: After a Databricks cluster restart, the JDBC driver entered
an infinite poll loop against the invalid operation handle. The root
cause was that `GetOperationStatus` returned `INVALID_HANDLE_STATUS` in
`TStatus.statusCode` but did not set `operationState`, so
`shouldContinuePolling()` kept returning `true`. With the default
`queryTimeout=0` (infinite), there was no safety net to break the loop.

The metadata polling loop (`fetchMetadataResults`) had a separate issue:
no timeout handler and no sleep between polls, meaning it could hammer
the server in a tight loop indefinitely.

SEA mode is not affected — it uses HTTP status codes (e.g. 404) for
invalid statements, which propagate as uncaught `RuntimeException`
rather than causing an infinite loop.

## Test plan
- [x] `testPollingThrowsOnInvalidHandleStatus` — SQL execution polling
detects `INVALID_HANDLE_STATUS` and throws
- [x] `testMetadataPollingThrowsOnInvalidHandleStatus` — metadata
polling detects `INVALID_HANDLE_STATUS` and throws
- [x] `testMetadataPollingTimesOut` — metadata polling respects timeout,
cancels operation, and throws
- [x] `testMetadataPollingWithSleepBetweenPolls` — verifies sleep delay
between metadata polls
- [x] All 47 tests in `DatabricksThriftAccessorTest` pass (43 existing +
4 new)


🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Gopal Lal <gopal.lal@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…atch RPC (#1277)

## Summary
- When multiple cloud fetch download threads detect expired presigned
URLs simultaneously, they each made independent FetchResults RPCs,
causing thread pool exhaustion under high concurrency
- Introduces a centralized coalescing mechanism in
`StreamingChunkProvider.getRefreshedLink()` that serializes refresh
requests under a lock, performs a single batch `fetchLinks()` call from
the lowest expired chunk offset, and distributes fresh links to all
expired chunks
- Adds `LinkRefresher` functional interface to decouple download tasks
from the coalescing logic, with fallback to single `refetchLink()` when
the batch doesn't include the target chunk

## Test plan
- [x] `StreamingChunkDownloadTaskTest` — 6/6 pass (uses `LinkRefresher`
mock)
- [x] `StreamingChunkProviderTest$LinkExpiryTests` — 9/9 pass (verify
batch coalescing behavior)
- [ ] Manual verification with high-concurrency cloud fetch workload to
confirm reduced FetchResults RPC count

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Gopal Lal <gopal.lal@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…e clusters (#1244)

## Description

Follow-up to #1199 ([ES-1717770]). The previous fix covered the case
where `FetchResults` returns an error status with `sqlState=57KD0`.
However, the interactive cluster path was still broken.

**Root cause:** When using interactive clusters with
`enableDirectResults=true`, the cluster can enforce its own server-side
query timeout and return `TIMEDOUT_STATE` directly in
`directResults.operationStatus` — before the client's polling loop ever
starts. Because `isErrorOperationState` did not include
`TIMEDOUT_STATE`, the driver:

1. Did not throw in `checkOperationStatusForErrors`
2. `shouldContinuePolling(TIMEDOUT_STATE)` returned false → polling loop
never started → `TimeoutHandler` never fired
3. Fell through to `executeFetchRequest` → server returned an error →
driver threw `DatabricksHttpException` instead of
`DatabricksTimeoutException`

The same gap also affects the polling path when `GetOperationStatus`
returns `TIMEDOUT_STATE` during polling.

**Fix:**
- Add `TIMEDOUT_STATE` to `isErrorOperationState`
- Throw `DatabricksTimeoutException` for `TIMEDOUT_STATE` in
`checkOperationStatusForErrors` regardless of whether `sqlState` is set
(interactive clusters do not always populate it)

## Testing

- `testTimedOutStateInDirectResultsThrowsTimeoutException` — Pavan's
exact repro: server returns `TIMEDOUT_STATE` in `directResults` before
polling starts
- `testTimedOutStateDuringPollingThrowsTimeoutException` — server
returns `TIMEDOUT_STATE` during polling

## Additional Notes

The original ES-1717770 verification test passed on both warehouse and
all-purpose cluster because `setQueryTimeout(1)` with a long-running
query caused the server to return `RUNNING_STATE` first (query still
in-flight), entering the polling loop where `TimeoutHandler` fired
correctly. Pavan's repro consistently hits the other path: the cluster's
own timeout fires first, returning `TIMEDOUT_STATE` directly, bypassing
the polling loop entirely.

[ES-1717770]:
https://databricks.atlassian.net/browse/ES-1717770?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Signed-off-by: Samikshya Chand <samikshya.chand@databricks.com>
Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>

## Summary

Fixes #1221 — `IdleConnectionEvictor` threads accumulate over time in
long-running services even when all JDBC connections are properly
closed.

**Root cause (two related bugs in
`DatabricksDriverFeatureFlagsContextFactory`):**

1. **Ref-count overcounting**: `getInstance()` incremented the ref-count
on every call, but `isTelemetryEnabled()` calls it repeatedly for the
same connection while `DatabricksConnection.close()` only calls
`removeInstance()` once. This meant the feature-flags context (and its
15-minute scheduler) never shut down after the last connection closed.

2. **Stale connection context → orphaned HTTP client**: The
feature-flags context is shared per-host but HTTP clients are scoped
per-connection UUID. The context stored the *first* connection's
`connectionContext`. When that connection closed, its HTTP client was
removed from the factory. On the next 15-min refresh,
`refreshAllFeatureFlags()` called
`DatabricksHttpClientFactory.getClient(staleContext)` — the UUID was
gone, so `computeIfAbsent` silently created a brand-new
`DatabricksHttpClient` (and a new `IdleConnectionEvictor` thread) that
nobody would ever clean up.

**Fix** (mirrors the pattern already used correctly in
`TelemetryClientFactory`):
- `FeatureFlagsContextHolder` now tracks connection UUIDs
(`ConcurrentHashMap<String, IDatabricksConnectionContext>`) instead of a
bare ref-count.
- `getInstance()` registers the UUID idempotently — no overcounting.
- `removeInstance()` removes the UUID; when the set is empty the context
shuts down.
- When the "owner" connection closes but others remain, the stored
`connectionContext` is atomically updated to another active connection
so future HTTP-client lookups resolve to a live client.

## Test plan

- [x] `DatabricksDriverFeatureFlagsContextFactoryTest` — updated
existing tests for new UUID-based semantics and added:
- `testMultipleCallsWithSameConnectionAreIdempotent` — single
connection, multiple `getInstance` calls, one `removeInstance` is
sufficient
- `testMultipleConnectionsRequireAllToClose` — two connections share
context; closing one keeps it alive, closing both shuts it down
- `testConnectionContextUpdatedWhenOwnerConnectionCloses` — verifies the
stored `connectionContext` switches to another connection when the owner
closes
- [x] All existing `DatabricksDriverFeatureFlagsContextTest` tests still
pass
- [x] Full build with `mvn test` passes

---------

Signed-off-by: Samikshya Chand <samikshya.chand@databricks.com>

## Description

Implements `getProcedures` and `getProcedureColumns` in
`DatabricksDatabaseMetaData` by querying `information_schema.routines`
and `information_schema.parameters` via SQL.

Unlike other metadata operations that use `SHOW` commands or Thrift
RPCs, these use direct SQL SELECT queries against `information_schema`
views. This works for both Thrift and SEA transports.

**getProcedures:**
- Queries `information_schema.routines` filtered by `routine_type =
'PROCEDURE'`
- Returns 9-column JDBC-spec result set (PROCEDURE_CAT, PROCEDURE_SCHEM,
PROCEDURE_NAME, reserved x3, REMARKS, PROCEDURE_TYPE, SPECIFIC_NAME)
- PROCEDURE_TYPE is always `procedureNoResult` (1)

**getProcedureColumns:**
- Queries `information_schema.parameters` JOINed with
`information_schema.routines` to filter for procedures
- Returns 20-column JDBC-spec result set with parameter metadata
- Maps `parameter_mode` (IN/OUT/INOUT) to JDBC COLUMN_TYPE constants
(1/4/2)
- Maps Databricks type names to `java.sql.Types` codes via existing
`getCode()`

**Catalog resolution:**
- NULL catalog → queries `system.information_schema.routines`
(cross-catalog)
- Specific catalog → queries `<catalog>.information_schema.routines`
- Empty string → returns empty result set

**Shared SQL builders** in `CommandConstants` eliminate duplication
between SDK and Thrift clients.

## Testing

**Unit tests** (`DatabricksMetadataSdkClientTest`):
- 4 parameterized tests for `listProcedures` SQL generation
(catalog+schema+name, null schema, null name, null catalog)
- 3 parameterized tests for `listProcedureColumns` SQL generation (all
filters, partial filters, all nulls)

**Integration test**
(`MetadataIntegrationTests#testGetProceduresAndProcedureColumns`):
- Creates a procedure `jdbc_test_compute_area(x DOUBLE, y DOUBLE, OUT
area DOUBLE)` with COMMENT
- Verifies `getProcedures` returns correct name, schema, catalog,
remarks, type
- Verifies `getProcedureColumns` returns 3 params with correct
COLUMN_TYPE (IN=1, OUT=4), DATA_TYPE (DOUBLE=8), ordinal positions
- Tests column name filtering
- Cleans up procedure after test
- WireMock stubs recorded for REPLAY mode

**Existing tests:** All 248 `DatabricksDatabaseMetaDataTest` + 51
`DatabricksMetadataSdkClientTest` pass.

## Additional Notes to the Reviewer

- `information_schema.parameters` is used (not `routine_columns`)
because `routine_columns` contains table-valued function output columns,
not procedure parameters.
- NULLABLE is always `procedureNullableUnknown` (2) since the server
does not track parameter nullability.
- When catalog is NULL, `system.information_schema` is queried which
requires system table access permissions. If the user lacks this
permission, the driver returns an empty result set. This will be
addressed server-side in a future release.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

## Description
- New version cut: 3.3.1
- Version bumped across all pom.xml files, Java source, tests, README,
and release notes
- NEXT_CHANGELOG entries moved to CHANGELOG.md and release-notes.txt

## Testing
- Version string tests updated for 3.3.1
- CI will validate build and tests

## Additional Notes to the Reviewer
Modeled after PR #1194 (release 3.2.1).

OVERRIDE_FREEZE=true

This pull request was AI-assisted by Isaac.

---------

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

## Summary
- Skip OWASP dependency check in the "Build dependencies" step of
`release.yml` and `release-thin.yml` by adding
`-Ddependency-check.skip=true`
- The deploy step already runs the OWASP check with the NVD API key —
running it in the build step was redundant and caused the v3.3.1 release
to fail (downloaded 338K NVD records without an API key, then hit a
false positive on CVE-2026-25087)

## Test plan
- [ ] After merging, delete the v3.3.1 release and tag, then re-create
from GitHub UI to re-trigger the release workflow
- [ ] Verify the release workflow completes successfully

NO_CHANGELOG=true

OVERRIDE_FREEZE=true

This pull request was AI-assisted by Isaac.

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

…ats (#1250)

## Summary
- **TIMESTAMP / TIMESTAMP_NTZ** fields inside complex types (ARRAY, MAP,
STRUCT) are serialized as epoch microseconds by Arrow. Added fallback to
convert epoch micros to `java.sql.Timestamp`. Also handles TIMESTAMP_NTZ
serialized as `[year,month,day,hour,min,sec]` component arrays.
- **BINARY** fields inside complex types are serialized as
base64-encoded strings by Arrow. Added base64 decoding in
`convertPrimitive()`.
- Added `TIMESTAMP_NTZ` case to `DatabricksStruct.convertSimpleValue()`
and `DatabricksArray.convertValue()` switch statements that were missing
it.
- Added server-format comments documenting how Arrow serializes each
type within nested structures.

Related: #1247, #1248

## Test plan
- [x] 7 new unit tests covering TIMESTAMP, TIMESTAMP_NTZ, and BINARY
across struct, array, and map containers
- [x] All 20 tests in `ComplexDataTypeParserTest` pass
- [x] Server output formats verified via E2E tests against real
Databricks warehouse
- [ ] CI passes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

#1285)

## Summary
- `generateMultiRowInsert()` now wraps column names with backticks when
reconstructing multi-row INSERT SQL
- This fixes `PARSE_SYNTAX_ERROR` for column names containing dots
(e.g., `col.name`) when `EnableBatchedInserts` is enabled
- Added test case for dotted column names

## Root Cause
`parseColumns()` strips backticks from column names, but
`generateMultiRowInsert()` joins them back without re-quoting. Unquoted
column names containing dots are then interpreted as schema/table
separators by the SQL parser.

## Test plan
- [x] Existing `InsertStatementParserTest` tests updated and passing
(18/18)
- [x] New `testGenerateMultiRowInsertWithDottedColumnNames` test added

Fixes #1284

---------

Signed-off-by: Oleksandr Shevchenko <oleksandr.shevchenko@datarobot.com>
Co-authored-by: Vikrant Puppala <vikrant.puppala@databricks.com>

… jar (#1294)

## Summary
- The `dependencyManagement` overrides for `commons-lang3` and `gson`
were only in `jdbc-core/pom.xml`, not inherited by `assembly-uber`
- The uber jar was bundling `commons-lang3:3.14.0` (from
`commons-configuration2` transitive dep) instead of `3.18.0`
- Moved overrides to parent `pom.xml` so all modules resolve safe
versions

Fixes #1293

## Test plan
- [x] `mvn dependency:tree -pl jdbc-core` shows `commons-lang3:3.18.0`
- [x] `mvn dependency:tree -pl assembly-uber` shows
`commons-lang3:3.18.0`
- [ ] CI unit tests pass

NO_CHANGELOG=true

Signed-off-by: Oleksandr Shevchenko <oleksandr.shevchenko@datarobot.com>
Co-authored-by: Samikshya Chand <148681192+samikshya-db@users.noreply.github.com>

## Summary
- Add `claude.yml` for `@claude` mentions on issues and PRs
- Add `claude-code-review.yml` for on-demand `/review` PR reviews
- Matches the workflow setup in `databricks-odbc`
- Uses `databricks-protected-runner-group` runner

NO_CHANGELOG=true

This pull request was AI-assisted by Isaac.

---------

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Co-authored-by: Samikshya Chand <148681192+samikshya-db@users.noreply.github.com>

…ck in release (#1296)

## Summary
- Added `owasp-suppressions.xml` to suppress CVE-2026-25087 (Apache
Arrow C++ only, false positive for Java Arrow libraries)
- Referenced suppression file in `jdbc-core/pom.xml` OWASP plugin config
- Re-enabled the OWASP dependency check in the release build step
(replaces `-Ddependency-check.skip=true` from #1286) and passes NVD API
key for fast NVD updates

Previously #1286 skipped the OWASP check entirely in the build step as a
quick fix for the v3.3.1 release. This PR restores the check with proper
suppression so real vulnerabilities are still caught during releases.

## Test plan
- [ ] Verify CI passes with the suppression file
- [ ] Verify the OWASP check runs during the release build step without
failing on Arrow CVE

NO_CHANGELOG=true

OVERRIDE_FREEZE=true

This pull request was AI-assisted by Isaac.

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

## Summary
- Replaced the hard-coded `1.20.0` version for `jts-core` in
`jdbc-core/pom.xml` with a `${jts-core.version}` property defined in the
parent `pom.xml`
- This ensures the version is centrally managed and cannot diverge
across modules (same class of issue as #1293 where `commons-lang3` and
`gson` resolved to different versions in the uber jar)

## Test plan
- [x] `mvn dependency:tree -pl jdbc-core` confirms `jts-core:1.20.0`
resolves correctly
- [ ] CI unit tests pass

NO_CHANGELOG=true

This pull request was AI-assisted by Isaac.

Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

## Summary

TLDR : update jdk8 instructions to include more details (so as to take
lesser time in jdk 8 branch update)

More detailed changes :

- Spotless: update from skip-profile approach to remove entirely (no
plugin, no pluginManagement, no version property)
- Add OWASP `dependency-check-maven` plugin removal step
- Add `coverageReport.yml` transformation (JDK 8, 80% threshold, remove
JVM17+ group filter)
- Add Java 9+ API replacement table (`List.of`, `Optional.isEmpty`,
`LocalDate.EPOCH`, `URLDecoder.decode(Charset)`, etc.)
- Add JDBC 4.3 test method removals (`enquoteLiteral`,
`enquoteIdentifier`)
- Add `IntervalConverter` `Duration.ofNanos(Long.MIN_VALUE)` test
removal with explanation
- Add `DatabricksDriverFeatureFlagsContext` hang fix guidance for JDK 8
- Add rule: only modify `coverageReport.yml`, keep all other workflow
files as-is
- Preserve existing TODOs with added context comments

NO_CHANGELOG=true

---------

Signed-off-by: Samikshya Chand <samikshya.chand@databricks.com>
Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>

Move attacker-controlled PR body and derived step outputs from inline
${{ }} expressions to env variables, preventing shell command injection
via malicious PR descriptions (e.g. PR #1290).

Co-authored-by: Isaac

## Description
<!-- Provide a brief summary of the changes made and the issue they aim
to address.-->

## Testing
<!-- Describe how the changes have been tested-->

## Additional Notes to the Reviewer
<!-- Share any additional context or insights that may help the reviewer
understand the changes better. This could include challenges faced,
limitations, or compromises made during the development process.
Also, mention any areas of the code that you would like the reviewer to
focus on specifically. -->

NO_CHANGELOG=true (not a user facing change)

Adds 37 new E2E integration tests to TransactionTests.java covering
gaps identified in MST + xDBC metadata audit (LC-13424, LC-13425,
LC-13427, LC-13428):

- executeUpdate/executeLargeUpdate/executeBatch in transactions
- DatabaseMetaData operations (getColumns, getTables, etc.) in active txn
- PreparedStatement metadata before/after execute in transaction
- MSTCheckRule-blocked SQL (SHOW/DESCRIBE/information_schema) in MST
- Allowed operations (setCatalog, setSchema, DESCRIBE TABLE) in MST
- Connection close with pending transaction (implicit rollback)
- DDL behavior in transactions (CREATE/DROP/ALTER TABLE)
- PreparedStatement parameterized DML in transactions
- Concurrent DDL + parameterized DML (stale schema)
- Edge cases: empty txn, read-only txn, holdability, timeout, retry

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

- Add ignoreTransactions=0 to JDBC URL to enable MST
- Fix testCloseConnectionWithPendingTransaction: accept both auto-commit
  and rollback behaviors on close
- Fix testGetFunctionsInsideActiveTransaction: catch known driver logging
  bug (IllegalFormatConversionException in getFunctions)

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

- setCatalog/setSchema/DESCRIBE TABLE are all blocked in MST via Thrift
  (SetCatalogCommand, SetNamespaceCommand, DescribeRelation) — tests now
  expect the exception
- getPrimaryKeys/getCrossReference may poison transaction in MST — wrapped
  in try-catch
- testParameterizedDMLAfterConcurrentAlterTable: catch driver logger bug
- testCommitWithoutActiveTransaction: remove message content assertion
- testTransactionContinuesAfterAllowedMetadataOp: use DML-only operations

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

- Add transactionTests.yml workflow to run TransactionTests and
  ExplicitTransactionStatementTests on push to main (same pattern as
  concurrencyExecutionTests.yml)
- Update both e2e test classes to read credentials from env vars
  (DATABRICKS_HOST, DATABRICKS_TOKEN, DATABRICKS_HTTP_PATH, etc.)
  with fallback to hardcoded defaults for local development
- Fake service replay mode not yet possible for transaction tests
  due to LC-13424 (SEA INLINE disposition bug with MST commands)

Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

…edentials"

This reverts commit dc758c7.